Initialization of Model Based Coding

Model based coding is a promising very low bit rate video coding technique. The idea behind model-based coding is to parameterize a talking head, and then to extract and transmit the parameters describing the facial movements, such as the facial expressions and the head motion to the receiver. At the receiver, the parameters are used to control face animation in order to reconstruct the talking head. Since only high-level animation parameters are required to drive face animation, very high compression can be achieved with this scheme. In this thesis, we focus on initialization of model based coding. The initialization problem of model based coding is to fit a generic face model onto a target face in the first video frame. Model fitting plays a key role and is the first step toward motion estimation and tracking in both model based coding and model-based tracking. Realizing that a fully automatic fitting is very difficult to achieve and that certain human intervention is inevitable, we propose a new strategy to do model fitting, pseudoautomatic fitting. The traditional initialization process is extended from the application stage only to include offline work. A manual fitting is done offline, where a generic face model is deformed into a personal face model. Another important work performed offline is to define, localize, and organize personal facial features. In the real application stage, only a simplified, automatic fitting operation is needed, detecting and matching the defined features from the first frame of application video. Different methods for implementing the initialization process are discussed in this thesis. The study of combining hardware based rendering with global search methods is presented first, followed by the study of using Dynamic Programming techniques to match facial features. The re-initialization problem is revisited and a new strategy is proposed for dealing with re-initialization.